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EXECUTIVE SUMMARY 



Introduction 

This report presents the findings of a study conducted by the Center for Research 
in Educational Policy (CREP) on the second year of implementation of the Small Schools 
Initiative at the Manual High School Complex. The major goal of this research was to 
evaluate perceived progress and outcomes at all three high schools. The design and 
methodology was oriented around the following research questions: 

1. What is the level of school climate at the three schools for the 2002-2003 school- 
year, and how does it compare to the first year of implementation (2001-2002)? 

2. What are the most positive and negative areas this year, as compared to last year? 

3. How did the three schools perform this year on the CSAP, relative to district and 
state scores, and as compared to last year? 

4. What is the level of implementation as viewed by teachers, and to what extent 
have perceptions changed this year, as compared to last year? 

5. What are the strengths and weaknesses of the Small Schools Initiative this year, as 
compared to last year? 

6. What are recommendations for strengthening the programs in the coming year? 

This report again begins with a brief overview of the three schools, a summary of 
findings for each of the three high schools, followed by an overall synthesis of the data, 
and final recommendations. Assessment instruments included: the School Climate 
Inventory (SCI®), the School Wide Program Teacher Questionnaire ( SWPTQ ®), the 
SWPTQ ® open-ended comments, and the School Observation Measure (SOM®). 

Results 

Arts and Cultural Studies High School 

SCI®. At Arts and Cultural Studies High School (ACSHS), there appeared to be a 
moderate decrease in overall school climate from last year to this year. ACSHS means 
for all seven dimensions were lower this year than last year, and lower than SCI® national 
norms. Again this year, the most positive scale dimensions for ACSHS were Leadership 
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and Instruction, and the lowest scale dimension was Order. Expectations were rated as 
mildly negative to neutral overall. Collaboration was rated as neutral. Environment was 
neutral/mildly positive, and Involvement was moderately positive overall. 

SWPTQ®. Few SWPTQ items garnered extreme support or disagreement. Like 
last year, ACSHS respondents were again mildly positive about Focus, neutral/mildly 
positive about Support, and mildly negative about Capacity /Resources. Reflecting slight 
improvement, respondents were neutral about Outcomes this year, whereas last year, they 
were mildly negative. However, respondents were strongly negative about Pedagogy, 
down sharply from last year, in which respondents were neutral. However, all five 
SWPTQ ® scale means were lower than national norms for the SWPTQ® . 

Teacher buy-in at the school appears to be positive, although less so than last 
year. Fewer agreed that they have a thorough understanding of the school’s program 
(40% this year, down from 53.8% last year). No one agreed that the school-wide 
program has changed classroom learning activities a great deal (70% disagreed). This 
reflected a sharp decline from last year, in which 38.5% agreed and just 53.8% disagreed. 

SOM. For the second year in a row, findings indicated that direct instruction was 
the most frequently observed instructional orientation, and that cooperative/collaborative 
learning was not observed more than occasionally during any of the visits. Three 
instructional strategies were observed more frequently this year than last year: use of 
higher level questioning strategies, project-based learning, and teacher acting as 
coach/facilitator. Additionally, three student activities were observed more frequently 
this year: independent seatwork, experiential hands-on learning, and student discussion. 

This year's academically focused class time was observed to be high on every 
visit, a strong improvement over last year. Likewise, the level of student attention, 
interest, and engagement was observed to be high 83.3% of the time, whereas last year, it 
was observed to be high only 40% of the time. 

CSAP Scores. Because of large variations in the percent not scored from year to 
year, CSAP scores were transformed to reflect only those students who were scored, and 
are noted in the text as the percent “of scored students.” The percent “of all students” 
refers to the original scores. For this summary, original scores (in which those “not 
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scored” were treated as a separate category and included as part of the total population) 
are reported, unless otherwise noted. 

Math was a concern for ACSHS’s 9 th and 10 th graders, and the percent of 
proficient and advanced scores continued to be lower than the district and state for both 
years. Only 1% scored proficient or advanced this year in either grade at ACSHS, as 
compared to only 3% at most last year. In the district, the percent of students scoring 
proficient or advanced was 9% for 9 th grade math this year, and 11% for 10 th grade math. 
Many scored unsatisfactory both this year and last year (73% scored unsatisfactory both 
this year and last year in 9 th grade math; and 68% scored unsatisfactory this year in 10 th 
grade math, although this was down from 81% last year. 

ACSHS’s reading scores appeared to be stronger than math, although 9 th grade 
reading scores dropped sharply. Approximately 10% of ACSHS’s 9 th graders scored 
proficient or advanced this year, representing a strong decline from 25% last year. For 
10 th grade reading, approximately 14% of all ACSHS students scored advanced or 
proficient in 10 th grade reading this year, marginally down from 17% last year. 
Transformed 10 th grade reading scores, however, indicated that approximately 17.2% of 
scored ACSHS students achieved advanced or proficient in 10 th grade reading this year, 
matching last year’s 17%. For both grades, the percent of students scoring unsatisfactory 
in reading (36% of all students; 46% of scored students) was up from 33% last year. The 
percent of proficient and advanced scores at ACSHS continued to be well below the 
district and state percentages. In the district, for example, the percent of students scoring 
proficient or advanced in 9 th grade reading this year was 37%; in 10 th grade reading, it 
was 43%. 

As for writing, no more than 5% scored proficient or advanced in either grade this 
year, and the percent of proficient and advanced scores continued to be lower than the 
district and state for both years. For the district, this year, 26% of DPS students scored 
advanced or proficient in 9 th grade writing; 30% scored advanced or proficient in 10 th 
grade writing. Both 9 th and 10 th grade ACSHS proficient and advanced scores were down 
from last year by at least eight percentage points. Further, the percent scoring 
unsatisfactory increased from last year for both grades (from 9% last year to 21% this 
year for 9 th grade writing; from 20% last year to 30% this year for 10 th grade writing). 
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Leadership High School 

SCI®. Faculty at Leadership High School (LHS) perceived overall school climate 
to be strongly positive ( M = 3.68). Leadership was the most positively rated dimension, 
followed by Instruction and Collaboration, all of which were perceived to be positive 
overall. The lowest rated dimension was Order, which was perceived as mildly negative. 
This year, Environment was perceived to be positive, a dramatic increase from last year’s 
perception of the Environment as mildly negative. Three of the seven SCI means 
(Leadership, Collaboration, and Instruction) matched or exceeded SCI ® national norms. 

SWPTQ®. Leadership High School teachers appeared to be generally positive on 
all scale means. All five means were higher than last year’s means, and three of the five 
(Support, Outcomes, and Focus) surpassed SWPTQ® national norms. This year, faculty 
at LHS were strongly positive about Support and Focus, up from mildly positive last 
year. They were also moderately positive about Outcomes again this year. As for 
Capacity/Resources, respondents were neutral/mildly positive this year, up from 
moderately negative last year. Finally, respondents were neutral about Pedagogy this 
year, up from mildly negative last year. 

Almost all teachers indicated that: a) the faculty and staff are committed to school 
goals, b) they have a thorough understanding of the school’s program, and c) they are 
generally supportive of the program. Further, many agreed that: a) teacher-student 
interaction is more positive, b) teachers spend more time collaborating to develop 
curriculum and plan instruction, and c) teachers are more involved in decision making 
than they were prior to implementation, representing increases over last year’s means. 

School Obser\’cition Measure. Direct instruction was again the most frequently 
observed instructional orientation again this year. Independent seatwork, sustained 
reading, and sustained writing were observed more frequently than last year. Other 
instructional strategies were observed about as frequently as last year. Project-based 
learning, cooperative learning, individual tutoring, use of higher- level questioning, and 
use of higher- level feedback were observed occasionally during one visit this year. 
Teacher acting as coach or facilitator was observed occasionally half of the time, and 
frequently or extensively the other half of the time. Technology was occasionally 
observed being used as a learning tool on one-third of the visits. 
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Academically focused class time was observed to be high two-thirds of the time 
(60% last year) and moderate the rest of the time (20% last year). Level of student 
attention, interest, and engagement was high one-half of the time (40% last year) and 
moderate the other half of the time (60% last year). 

CSAP Scores. Math scores continued to be a concern at LHS. No more than 2% 
scored proficient or advanced this year, in either grade. Last year, only 4% scored 
proficient or advanced in 9 th grade; none did in 10 th grade. For 9 th grade math, although 
many (65%) scored unsatisfactory this year, the percent was down from last year (77%), 
reflecting some improvement. For 10 th grade math, most (84%) scored unsatisfactory 
again this year, and the percent was up from last year (71%). The percent of LHS 
students scoring proficient or advanced continued to be below district and state 
percentages. (For the district, 9% scored proficient or advanced this year in 9 th grade 
math; 11% scored proficient or advanced in 10 th grade math this year.) 

For reading, there appeared to be sharp contrasts between 9 th and 10 th grades. For 
9 th grade, the percent scoring proficient or advanced this year (14%) dropped well below 
last year (28%) and continued to be well below district (37% this year) and state levels. 
For 10 th grade reading, the percent of LHS students scoring proficient or advanced this 
year (36%) was sharply up compared to last year (15%). Although the percent scoring 
proficient or advanced in 10 th grade reading was still below district percentages this year 
(43%), the sharp gains for LHS this year served to narrow the gap between LHS and the 
district. For both 9 th and 10 th grades, the percent of LHS students scoring unsatisfactory 
(21% for 9 th and 28% for 10 th ) this year appeared to remain relatively stable from last 
year (23% for 9 th and 26% for 10 th ). 

For writing, this year, the percent scoring proficient or advanced (10% for 9 th 
grade; 13% for 10 th grade) was similar to last year for 9 th grade writing (12%), and 
improved from last year for 10 th grade writing (3%). Nevertheless, no more than 13% 
scored proficient or advanced this year in writing for either grade. On the other hand, no 
more than 20% scored unsatisfactory for either grade. As for unsatisfactory scores, there 
appeared to be some improvement in the 9 th grade, with scores dropping this year (from 
18% last year to 10% this year). For the 10 th grade, however, the percent scoring 
unsatisfactory rose, from 11% last year to 20% this year. The percent of LHS students 
scoring proficient or advanced in writing for both grades continued to be well below the 
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district and state percentages. (DPS proficient or advanced scores ranged from 26% to 
30% both years for 9 th and 10 th grade writing.) 

Millennium Quest High School 

SCI®. Faculty perceived the overall school climate at Millennium Quest High 
School (MQHS) to be mildly positive this year, slightly higher than last year’s mean 
rating. As was the case at ACSHS and at LHS, faculty at MQHS reported the most 
positive dimension to be Leadership and Instruction. The lowest-rated dimension was 
Order, which has continued to develop into a very strong concern. Again this year, all 
SCI means for MQHS were below the SCI ® national norms. 

SWPTQ®. Only eight of the 20 Millennium Quest teachers completed the 
SWPTQ ® this year as compared to 18 of 19 last year. Thus, strong caution is warranted 
when interpreting Millennium Quest’s year-to-year findings. This year, all five scales 
(i.e., Support, Capacity /Resources, Pedagogy, Outcomes, Focus) were perceived as either 
neutral or negative. Last year, faculty were moderately to strongly negative on every 
scale. Despite the fact that none of the scales was perceived as positive and all were 
below national norms, all but one of the scale means (i.e., Capacity/Resources) were 
higher this year than last year. For example, MQHS respondents were neutral about 
Support and Pedagogy this year, which was improved from last year, when they were 
viewed as moderately negative. Perceptions of Capacity/Resources, however, were 
extremely negative, and represented a decline from last year. On benchmark ratings, 
teachers indicated (again this year) that reallocation of resources to support program 
implementation has either not happened or has been done for some components but not 
others. 

School Observation Measure. Direct instruction was the most frequently 
observed instructional orientation again this year. Individual tutoring and experiential 
learning were observed slightly more frequently this year. Cooperative/collaborative 
learning and higher-level instructional feedback were observed less frequently this year. 
Similar to last year, teacher acting as coach or facilitator was observed occasionally one- 
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third of the time, and frequently one-half of the time. Again this year, independent 
seatwork was observed either occasionally or frequently during each of the six visits. 

Both summary items suggested apparent declines from last year’s observations. 
This year, academically focused class time was observed to be moderate most of the 
time, whereas last year, it was observed to be high most of the time. Moreover, this year, 
the overall level of student attention, interest, and engagement was low two-thirds of the 
time, and moderate the rest of the time. Last year, it was moderate most of the time. 

CSAP Scores. Math appeared to be a strong concern for MQHS this year. Only 
1% scored proficient or advanced in either grade (as compared to 2% last year for 9 th 
grade, and 4% last year for 10 th grade). Most scored unsatisfactory again this year, with 
this year’s scores (89% for 9 th grade and 78% for 10 th grade math) higher than last year’s 
scores (77% for 9 th grade and 65% for 10 th grade math). The percent of MQHS students 
scoring proficient or advanced fell well below the district and state scores for both 
grades. (DPS proficient or advanced scores ranged from 9% to 12% for both grades this 
year and last year.) 

MQHS’s reading scores appeared to be considerably strong, as compared to math. 
One-fourth (25%) of MQHS’s 9 th grade students scored proficient or advanced this year 
in reading; another 26% scored unsatisfactory. More than one-third (35%) of the 10 th 
grade students scored proficient or advanced in reading; another 35% scored 
unsatisfactory. For both grades, the percent of MQHS students scoring proficient or 
advanced was similar to last year (26% for 9 th grade reading; 39% for 10 th grade reading), 
albeit marginally lower this year. The percent scoring unsatisfactory was down slightly 
this year from last year for the 9 th grade (31%), although it was marginally higher for the 
10 th grade (33%). This year, for both grades, the percent of MQHS students scoring 
proficient or advanced was well below district and state percentages. (DPS proficient or 
advanced scores ranged from 37% to 43% for both grades this year and last year.) Last 
year, however, the percent scoring proficient or advanced in 10 th grade reading was on 
par with the district (39%). 

The percent of students scoring proficient or advanced in writing was down from 
last year for both grades. For 9 th grade writing, about 7% of MQHS students scored 
advanced or proficient this year, down from 16% last year. For 10 th grade writing this 
year, 13% of MQHS students scored advanced or proficient in 10 th grade writing, down 



7 




from 21% last year. Moreover, the percent scoring unsatisfactory was higher this year for 
both grades, particularly so for the 10 th grade, in which 35% scored unsatisfactory, 
sharply higher than the 13% who scored unsatisfactory last year. Finally, the percent of 
MQHS students scoring proficient or advanced in writing (for both grades) were well 
below the district and state percentages. (DPS proficient or advanced scores ranged from 
26% to 30% for both grades this year and last year.) 

CONCLUSIONS 

This section will summarize the study’s findings as they relate to each of the five 
research questions. 

1. What is the level of school climate at the three schools for the 2002-2003 school- 
year, and how does it compare to the first year of implementation (2001-2002)? 

This year, overall school climate appeared to be neutral at Arts and Cultural 
Studies High School, strongly positive at Leadership High School, and mildly positive at 
Millennium Quest High School. At ACSHS, overall perceptions of school climate 
became more negative, as reflected by a moderate decrease in the mean for overall school 
climate, which was relatively positive last year. At LHS, school climate perceptions 
showed strong improvement, as indicated by a strong increase in the school climate 
mean, which was mildly positive last year. Finally, at MQHS, perceptions of school 
climate showed marginal increases over the slightly positive levels of the previous year. 

2. What are the most positive and negative areas this year, as compared to last 
year? 

Most positive areas. Like last year, the most positive school climate dimensions 
at all three schools were Leadership and Instruction, both of which were perceived as 
strongly positive overall. On the SWPTQ e , one positive area at both ACSHS and LHS 
was Focus (i.e., the extent to which elements of the school educational program are 
integrated, evaluated, and supported by school stakeholders). 

Teachers’ perceptions of Outcomes appeared to be another area that has shown 
improvement at ACSHS and MQHS, and has remained strong at the other. Other 
strengths, based on teacher comments, included the “small class size and personalization” 
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again this year, as well as the improved student confidence and motivation to learn, better 
teacher- student relationships, continued improvement efforts, integrated learning, 
assistance from community and businesses, and effective outreach, workshops, and 
career internships. 

Most negative areas. At all three schools, the most negatively perceived climate 
area again this year was Order, with this year’s aggregate mean being almost one full 
point (based on a five-point scale) below the national norm. For all three schools, Order 
was also the dimension that showed the steepest decline for any SCI dimension from 
Year 1 to Year 2. 

Another limitation appeared to be teachers’ perceptions of Capacity/Resources in 
two of the schools. Parent involvement, professional development, lack of teacher 
understanding of the program, and lack of buy-in appeared to be additional concerns 
again this year. Further weaknesses, according to negative comments, included: 
scheduling problems, lack of appropriate placement of students in art and music classes, 
lack of communication, and lack of a clearly defined school- wide program. 



3. How did the three schools perform this year on the CSAP, relative to district 
and state scores, and as compared to last year? 

Math scores appeared to be a weakness at all three schools. This year, no more 
than 2% scored proficient or advanced this year, in either grade at any of the three 
schools. The percent of proficient and advanced scores continued to be lower than the 
district and state for both years at all three schools. 

Reading scores appeared to be relatively strong, compared to math. However, the 
percent of 9 th graders scoring proficient or advanced dropped well below last year at 
ACSHS (10% this year, from 25% last year) and at LHS (14% this year, from 28% last 
year); at MQHS, scores were marginally lower this year (25% this year, from 26% last 
year). The percent of 10 th graders scoring proficient or advanced in reading was similar to 
(albeit marginally down from) last year at ACSHS (14% this year, from 17% last year) 
and at MQHS (35% this year, from 39% last year), and was sharply higher this year at 
LHS (36% this year, from 15% last year). The percent scoring proficient or advanced was 
well below district and state percentages for both grades at all three schools this year, 
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although 10 th grade gains at LHS narrowed the gap between LHS and the district this 
year. 

As for writing, no more than 13% scored proficient or advanced in either grade 
this year at any of the three schools. At LHS, the percent of students scoring proficient 
and advanced (10%) were similar to albeit marginally lower than last year (12%) for 9 th 
grade writing, and improved from last year for 10 th grade writing (13% this year, from 
3% last year). At ACSHS and at MQHS, the percent of students scoring proficient and 
advanced (no more than 5% this year for either grade at ACSHS; 7% for 9 th grade at 
MQHS; 13% for 10 th grade at MQHS) were down from last year in both grades by 
approximately 8 to 12 percentage points. At all three schools, for both ninth and tenth 
grades, the percent of proficient and advanced scores continued to be lower than the 
district and state for both years. 

4. What is the level of implementation as viewed by teachers, and to what extent 
have perceptions changed this year, as compared to last year? 

At MQHS and at LHS, the perceived level of implementation of research-based 
instructional strategies appears to have improved. Furthermore, at ACSHS and at LHS, 
respondents most commonly reported that curriculum redesign has been planned and that 
implementation has begun, with the rest indicating that curriculum redesign is being 
implemented in most grades and subject areas. There may also be residual issues of 
autonomy, which may still be impeding full implementation. 

5. What are the strengths of the Small Schools Initiative this year, as compared to 
last year? 

This year, like last year, the increase in personalization and/or closer staff- 
student-principal relationships was one of the more commonly mentioned positive 
teacher comments about the Small Schools Initiative. Indeed, at LHS and at MQHS, 
most agreed that people really care about each other. Further SWPTQ evidence of 
improved interactions between students and teachers as a result of the Small Schools 
Initiative was indicated at LHS only, where many agreed that student-teacher interactions 
are more positive because of the school-wide program. At ACSHS and MQHS, however, 
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less than half agreed that student-teacher interactions are more positive because of the 
school-wide program. 



6. What are the weaknesses of the Small Schools Initiative this year, as compared 
to last year? 

Like last year, many of this year’s difficulties pertained to limited resources that 
were shared between the schools. This issue of resources may be of particular concern 
given DPS’s financial troubles, which have recently resulted in extreme budget and staff 
cuts. On the other hand, as was posited last year, sharing resources, offerings, and 
curriculum ideas, may ultimately prove advantageous toward fostering inter-school 
collaboration. 

This year, lack of understanding and buy-in were again concerns at ACSHS and at 
MQHS, based on survey findings. At LHS, however, almost all agreed that teachers are 
generally supportive of the program, that the faculty and staff are committed to school 
goals and that they have a thorough understanding of the school's program. 



7. What are recommendations for strengthening the programs in the coming year? 

As the schools move into their third year of implementation, several important 
challenges may be faced, including: (a) weathering the effects of DPS budget cuts, 
including staff reductions, (b) easing tensions pertaining to sharing of resources, (c) 
developing a stronger emphasis on order and student discipline, and (d) improving 
faculty and staff understanding of and commitment to the school-wide program. 

At the same time, the schools should capitalize on the strengths that each has 
fostered, including: (a) continuing to increase personalization and improved relationships 
within the school community, (b) continuing strong leadership and instruction, (c) 
nurturing efforts to improve outcomes, and (d) maintaining the strong overall focus. 

The upcoming third year is critical to the success of Manual’s Small School’s 
Initiative. Based on the most common key needs of the schools, the following 
suggestions may assist in these areas. 
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• Schools should continue to use FEPSI and other data to monitor progress in 
achieving implementation and outcome goals. Schools need to use the 
benchmarking process regularly to ensure that all teachers understand how the 
program components are designed to work together to accomplish school reform. 

• Continue to develop and clarify school identities, in order to reflect the goals of 
each school. Develop and implement processes to increase teacher buy-in to the 
school identities and the Small High Schools initiative in general. 

• Continuing financial and personnel resources may be of concern. Identify which 
aspects of the Small Schools Initiative have had the biggest impact or will in the 
future, and investigate additional sources of funding to continue implementation 
in these areas or to address new areas of need. 

• Schools need to meet with key stakeholders (including representatives from the 
Small High Schools initiative) to determine needs and preferences, and identify 
structures, for sharing resources. Identify the extent to which and where the three 
schools can mutually benefit from appropriate and/or recommended pooling of 
resources. 

• Continue efforts to improve student behavior, motivation, and involvement. 
Encourage consistent enforcement of rules for student behavior and appropriate 
discipline. Elicit the assistance of the students, parents, and faculty and staff to 
brainstorm innovative ways to involve students. 

• Strategies that have produced strong positive climate indicators need to be 
maintained and possibly strengthened in order to ensure a supportive and 
nurturing organizational structure for teaching and learning. 
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EVALUATION REPORT: 

YEAR 2 IMPLEMENTATION OF MANUAL’S SMALL SCHOOLS INITIATIVE 

Introduction 

Beginning in the 2001-2002 school year, the large urban high school formerly 
known as Manual separated into three smaller high schools (grades 9-12): Arts and 
Cultural Studies High School (ACSHS), Leadership High School (LHS), and Millennium 
Quest High School (MQHS). This report presents the findings of a study conducted by 
the Center for Research in Educational Policy (CREP) on the three schools’ second year 
of implementation of the Small Schools Initiative. 

The major goal of this research was to evaluate perceived progress and outcomes 
at all three high schools. The design and methodology, to be described below, was 
oriented around the following research questions: 

1. What is the level of school climate at the three schools for the 2002-2003 school- 
year, and how does it compare to the first year of implementation (2001-2002)? 

2. What are the most positive and negative areas this year, as compared to last year? 

3. How did the three schools perform this year on the CSAP, relative to district and 
state scores, and as compared to last year and Manual’s pre-implementation year? 

4. What is the level of implementation as viewed by teachers, and to what extent 
have perceptions changed this year, as compared to last year? 

5. What are the strengths and weaknesses of the Small Schools Initiative this year, as 
compared to last year? 

6. What are recommendations for strengthening the programs in the coming year? 

This report again begins with a brief overview of the three schools, a summary of 
findings for each of the three high schools, followed by an overall synthesis of the data, 
and final recommendations. Assessment instruments included: the School Climate 
Inventory (SCI®), the School Wide Program Teacher Questionnaire ( SWPTQ ®), the 
SWPTQ ® open-ended comments, and the School Observation Measure (SOM®). 
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CREP Instrumentation 

The present section provides descriptions of the instrumentation used for data 
collection in the second year of this study. 

School Climate Inventory (SCI®). The main purpose of the School Climate 
Inventory is to assess impacts of reform initiatives in relation to seven dimensions 
logically and empirically linked with factors associated with effective school 
organizational climates (Butler & Alberg, 1991). The inventory contains 49 items, with 7 
items comprising each scale. Responses are scored through the use of Likert-type ratings 
ranging from strong disagreement (1) to strong agreement (5). Face validity of the school 
climate items and logical ordering of the items by scales were established during the 
development of the inventory (Butler & Alberg, 1991). Subsequent analysis of responses 
collected through administration of the inventory in a variety of school sites substantiated 
validity of the items. Scale descriptions and current internal reliability coefficients on the 
seven scales of the inventory, obtained using Cronbach’s alpha, are as follows: 

• Order. The extent to which the environment is ordered and appropriate student 
behaviors are present (a=.8394). 

• Leadership : The extent to which the administration provides instructional 
leadership (a=.8345). 

• Environment : The extent to which positive learning environments exist 
(a=.8094). 

• Involvement-. The extent to which parents and the community are involved in the 
school (a=.7582). 

• Instruction: The extent to which the instructional program is well developed and 
implemented (a=.7453). 

• Expectations : The extent to which students are expected to leam and be 
responsible (a=.7275). 

• Collaboration-. The extent to which the administration, faculty, and students 
cooperate and participate in problem solving (a=.7417). 

School-Wide Program Teacher Questionnaire (SWPTQ®). Teachers at each 
school were asked to complete the SWPTQ , which contains 28 items to which teachers 
respond using a five-point Likert-type scale ranging from strongly disagree (1) to 
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strongly agree (5). Among the areas assessed were professional development, impacts 
on students, changes in teaching, support for the program, effects on technology use, and 
involvement of parents and the community. Responses are reported as the percentages of 
teachers agreeing (i.e., marking either “agree” or “strongly agree”), disagreeing (i.e., 
marking either “disagree” or “strongly disagree”), or expressing no opinion (i.e., marking 
“neutral”). The scales are as follows: 

• Support (Questions 1-4, 27) - The extent to which the school receives effective 
professional development and support to implement its school- wide program. 

• Capacity/Resources (Questions 5-8) - The extent to which planning time, 
materials, technology, and faculty are available at the school. 

• Pedagogy (Questions 9-13) - The extent to which classroom practices, materials, 
and technology use have changed at the school. 

• Outcomes (Questions 14-21, 23) - The extent to which positive student, faculty, 
and parent/community outcomes have occurred as a result of SWP. 

• Focus (Questions 22, 24-26, 28) - The extent to which elements of the school 
educational program are integrated, evaluated, and supported by school 
stakeholders. 

© 

SWPTQ Open-Ended Comments and Benchmark Items. Teachers were asked to 
respond to two open-ended questions: "What do you view as positive aspects of your 
school’s SWP?" and "What do you view as negative aspects of your school’s SWP?" 
These responses were then transcribed verbatim from the questionnaire. Teachers were 
also asked to indicate their perceived progress toward six general benchmark goals. 
Responses are reported as the percentage of teachers selecting each response option for 
the six benchmark items. 

School Obserx’ation Measure ® (SOM 0 ). Trained, unbiased observers conducted 
targeted classroom observations, using the School Observation Measure® (SOM®). The 
standard SOM procedure involves observers’ visiting 10-12 randomly selected 
classrooms, for 15 minutes each, during a three-hour visitation period. The observer 
examines classroom events and activities descriptively, not judgmentally. Notes are 
taken relative to the use or nonuse of 24 instructional strategies, such as cooperative 
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learning, direct instruction, and technology. At the conclusion of the three -hour visit, the 
observer summarizes the frequency with which each of the strategies was observed across 
all classes in general on the SOM Data Summary form. The frequency is recorded via a 
5-point rubric that ranges from (0) Not Observed to (4) Extensively. 

CSAP Achievement Data 

Longitudinal student achievement data from the Colorado Student Assessment 
Program (CSAP) were provided by grade (9 th and 10 th ) and subject area (Math, Reading, 
and Writing). Students’ scores ranged from advanced to proficient to partially proficient 
to unsatisfactory. For this report, results are compared across the past three years, with 
Manual’s pre-implementation year findings (2000-2001) as a baseline. Additionally, each 
school’s results are compared to district and state results. 

Demographics 

The following data were obtained from school demographic information provided 
by the Colorado Small Schools Initiative (CSSI). 1 

Student Enrollment Profile 

Over the past year, ACSHS experienced a 26% increase in the number of students 
from 2001 to 2002 (see Table 1). LHS experienced a 12% increase in enrollment, and 
Millennium Quest experienced no changes. 

Table 1. Number of Teachers, Students, and Teacher-to-Student Ratios for 
Manual Complex: Year 1 and Year 2 Implementation 





2001-2002 


2002-2003 


Teachers 


Students 


Teacher: Student 
Ratio 


Teachers 


Students 


Teacher: Student 
Ratio 


Arts and Cultural Studies 


28 


353 


1: 12.6 


30 


428 


1: 14.3 


Leadership 


25 


307 


1: 12.3 


24 


341 


1: 14.2 


Millennium Quest 


19 


322 


1: 16.9 


20 


322 


1: 16.1 



1 Note that other sources have provided slightly varying numbers of faculty. According to Santo Nicotera’s 
report, in Year 1, ACSHS had 25 teachers, LHS had 21, and MQHS had 18.5. According to McQuillan’s 
Year 2 report, Principal Lewis said, “I have 22.5 teachers in Millennium Quest and I do all of their 
evaluations.” 
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Student Ethnicity 

As shown in Table 2, ethnicity breakdowns remained fairly constant with ACSHS 
and MQHS being majority Hispanic, and LHS being majority African American. 

Table 2. Hispanic and African-American Ethnicity Profiles for Manual 
Complex: Year 1 and Year 2 Implementation 





2001-2002 


2002-2003 


Hispanic 


African- 

American 


Hispanic 


African- 

American 


Arts and Cultural Studies 


82.1% 


16.6% 


87.6% 


1 1 .2% 


Leadership 


45.2% 


50.5% 


47.5% 


48.7% 


Millennium Quest 


61.1% 


33.9% 


59.6% 


36.6% 



Note. Caucasian, Asian, and Native American ethnicity profiles accounted for less than 5% of the students 
at each school. 



Study Participants 

Of the 30 teachers at ACSHS, 2 1 1 completed the SCI ® and 10 completed the 
SWPTQ ®, for response rates of only 36.7% and 33.3%, respectively. This was down 
from last year, in which 13 of 28 teachers completed both the SCI ® and SWPTQ ® , for a 
response rate of 46.4%. 

Of the 24 teachers at LHS, 15 completed the SCI®, for a response rate of 62.5%. 
This represented a large increase from last year, in which only six participants (out of 25) 
completed the SCI®, for a response rate of only 24%. On the SWPTQ®, 13 respondents 
completed the survey both this year and last year, for response rates of 54.2% and 52%, 
this year and last year respectively. 

Finally, of the 20 teachers at Millennium Quest High School, only 8 faculties 
completed the SCI® and the SWPTQ®, for a response rate of 40%. This was a substantial 
decrease from the 18 of 19 teachers responding last year. (Last year’s response rate for 
both surveys was 94.7%.) 



2 

“ Here again, the number of faculty at the three schools were obtained from the school demographic 
information provided by CSSI. 
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